System and method for image tamper detection via thumbnail hiding

ABSTRACT

A semi-fragile image tamper detection system and method based on thumbnail hiding and which signifies the image areas in which tampering has occurred, is provided. A thumbnail is defined as a low-resolution version of an entire image or the significant features of an entire image. Thumbnail hiding is chosen to be resilient to effects of both image compression and low levels of transmission channel noise. The thumbnail computation and embedding techniques are chosen such that the resultant marked image corresponds in size and dynamic range to the original image.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to the field of digital imagetamper detection. More particularly, the invention deals with a systemand method of digital image tamper detection that can effectively detectimage manipulations and their locations within the image but is tolerantof the effects of low-level image compression and additive channelnoise, i.e., a semi-fragile technique.

[0003] 2. Description of Related Art

[0004] Image tamper detection techniques are well known in the art as amethod used to detect modification of an image from its original state.

[0005] A range of tamper detection techniques has been described,varying in degree from fragile techniques, that can detect the mostminute alterations of an image, to more robust techniques, that aretolerant of significant modifications of an image. Semi-fragiletechniques lie between these extremes and are capable of detectingsignificant modifications while, at the same time, tolerating minoralterations, such as those induced by low levels of lossy compression orchannel noise.

[0006] Approaches to semi-fragile tamper detection have been proposed inthe literature, all of which employ some variation of an extracted imagesignature. The concept of semi-fragile tamper detection is described byM. Schneider and S. Chang, Proceedings of the IEEE InternationalConference on Image Processing, pages 227-230, Lausanne, Switzerland,September 1996. Schneider and Chang proposed a semi-fragile technique oftamper detection that extracts a content-based signature of the image asintensity histograms from blocks of image pixels. This content-basedsignature is then embedded within the original image using any one of avariety of known watermarking techniques. This technique is robust to acertain level of lossy image compression.

[0007] Another scheme for semi-fragile tamper detection is described byL. Xie and G. Arce in “A Watermark for Digital Images,” Proceedings ofthe IEEE International Conference on Image Processing, Chicago, Ill.,October 1998. Xie and Arce describe a scheme of embedding edgeinformation extracted from the original image within the low-ordercoefficients of a wavelet transform.

[0008] S. Bhattacharjee and M. Kutter, in “Compression Tolerant ImageAuthentication,” Proceedings of the IEEE International Conference onImage Processing, Chicago, Ill., October 1998, describe anothercompression tolerant tamper-detection scheme. Their scheme extractsperceptually interesting feature points that are not embedded within theimage but maintained separately.

[0009] In M. Wu and B. Liu, “Watermarking for Image Authentication,”Proceedings of the IEEE International Conference on Image Processing,Chicago, Ill., October 1998, a technique is presented that invisiblyembeds within the original image a visually meaningful watermark, alongwith a set of simple features, by altering coefficients that are indexedinto a look-up table of frequency domain coefficients.

[0010] In addition to approaches that are described in the literature,several patents teach devices and methods for image tamper detectionand/or authentication. An image tamper detection and/or authenticationprocess is disclosed by Ward, in U.S. Pat. No. 5,760,386, in which theimage of the holder of an identification document is stored, in highlycompressed form, in a magnetic medium dispersed invisibly within avisible print of the holder that is part of the document or within thebody of the identification document itself. When the document is scannedmagnetically at the place where identification is being authenticated,this image is decompressed and displayed in order to confirm that theholder of the document is the person shown and that the document has notbeen tampered with. The entire content of the original image is used inWard's process. This tamper detection approach is not restricted tocertain acquisition devices.

[0011] Shimizu, et al., U.S. Pat. No. 6,005,936, disclose a digitalcamera device and method for embedding an extracted image signature in adigital image. In response to the digital signal of the image, a digitalcamera having a region-dividing unit divides the digital image intofirst and second regions. Authentication information is generated in theform of a hash value from data in the first region and this informationis then encrypted using a secret key, which differs for each camera andis held within the camera itself. The encrypted hash value is thenembedded in the second image region, and the first and second regionsare then combined to form a combined image. An alteration detectingsystem is also disclosed in which an authenticator uses a public keycorresponding to the camera's secret key to decrypt the hash value. Theoriginal division of the image is detected and the hash value iscalculated anew and compared with the decrypted hash value to accomplishauthentication of the image. This approach uses only part of theoriginal image for authentication, i.e., the first image region, and isrestricted to a particular image acquisition device, i.e., speciallyequipped digital cameras.

[0012] Friedman, U.S. Pat. No. 5,499,294, discloses a digital cameraequipped with a processor for generating a digital signature by hashingan image file using a predetermined algorithm and encrypting the hashedfile with a private key stored in the camera's processor. The encryptedimage may be decrypted using a public key that is stored in the camera'shousing. Both the image file and the digital signature are storedindividually (not embedded one within the other) but in such a way thatthey will be available together. For authentication purposes only, thepublic key is used to decrypt the digital signature for comparison witha newly created hash of the entire image. This tamper detection approachis restricted to certain acquisition devices, i.e., specially equippeddigital cameras. It is also possible for the digital signature and theoriginal image file to be separated from one another by a maliciousattacker intent on defeating this security technique.

[0013] Schipper, et al., U.S. Pat. No. 5,987,136, disclose an apparatusfor producing a self-authenticating visual image of a selected view,using a digital image forming means such as a digital camera, togetherwith a position determining system that provides position information.The position information is incorporated in the digital image byaltering pixel bit values in a selected authentication pattern of thepixel array associated with the digital image. A set of polygons is usedas the authentication pattern, wherein each polygon contains at leastone pixel from the selected subset of the pixel array. Theauthentication pattern is represented by a key or ordered sequence ofkeys and may be encrypted and either stored with or as a part of thedigital image. This tamper detection approach is restricted to certainacquisition devices, i.e., specially equipped digital cameras. If theauthentication pattern is stored separately from the image, there is thepossibility of its destruction by a malicious attacker.

[0014] Squilla, et al., U.S. Pat. No. 5,898,779, disclose a public keyencryption system for authenticating an image using a digital camerawhich has a private key embedded in it that is unique to the digitalcamera. A known public key, uniquely based on the private key, is usedto decrypt digital image data encrypted with the private key in order toestablish authenticity of an image. A digital signature is produced bythe camera using one or more patterns taken from an active area of theimage and input to a predetermined hash function. The hashed output isthen encrypted employing the embedded private key. The digital signatureand location of the active area are stored together with the image butnot within the image. This tamper detection approach is restricted tocertain acquisition devices, i.e., specially equipped digital cameras,and is subject to destruction of both the digital signature and thelocation of the active area by a malicious attacker.

[0015] Murphy, et al., U.S. Pat. No. 5,799,082, disclose an apparatusfor capturing and authenticating a visual image of a selected view,using a digital image forming means, such as a digital camera, togetherwith a position determining system that provides position information.Any suitable overlay pattern of a selected subset of the array of imagepixels may be used to incorporate the position information in thedigital image by altering the pixels lying within this pattern. Theposition information may be encrypted, using an encryption key based onposition information, and may be stored as part of the digital image.Optionally, position information includes the distance from the digitalcamera to the selected object in the selected view. Murphy's tamperdetection approach is restricted to certain image acquisition devices,i.e., specially equipped digital cameras, and the authenticationinformation is susceptible to destruction by a malicious attacker if itis stored separately and not embedded in the image.

[0016] Tamper detection techniques known in the art have variousvulnerabilities and limitations. In several techniques the tamperdetection information is developed using only a part of the image and ifa different part of the image is modified, this approach cannot detectthe tampering. In other techniques the tamper detection information iseasily separated from the image because it is stored with but separatefrom the image. Many known techniques are restricted to certainacquisition devices. Many known techniques cannot indicate where in theimage the detected tampering has occurred.

SUMMARY OF THE INVENTION

[0017] According to the present invention, there is provided asemi-fragile tamper detection system and method that overcomes theabove-noted prior art shortcomings. More particularly, the semi-fragiletamper detection system and method of the present invention hides alow-resolution version of the entire original image within the originalimage, is acquisition device independent, and signifies the image areasin which detected tampering has occurred.

[0018] The semi-fragile tamper detection scheme of the present inventionuses data hiding techniques to embed a low-resolution version of anoriginal image within the image itself to form a marked image. Alow-resolution version of an image is defined as a thumbnail of thatimage. Any data hiding technique can be used that is resilient to theeffects of both image compression and low levels of transmission channelnoise.

[0019] With proper selection of a data hiding technique, the hiddeninformation (thumbnail) is recoverable even if the image has beencompressed via low levels of image compression, e.g., JPEG, or exposedto additive transmission channel noise. Gross alterations in atransmitted marked image may then be detected by extracting the embeddedthumbnail and comparing it with a newly computed thumbnail of thereceived marked image. If the two thumbnails are sufficiently similar,it can be concluded that no tampering has occurred and the image isauthenticated. Alternatively, if the thumbnails are not sufficientlysimilar, it can be concluded that the image has been manipulated and atamper alert can be issued.

[0020] An image thumbnail can be constructed in a number of ways,including, low-pass filtering followed by decimation and waveletdecomposition. Data hiding can be accomplished by a variety of datahiding techniques such as Spread Spectrum Image Steganography (SSIS).

BRIEF DESCRIPTION OF THE DRAWINGS

[0021]FIG. 1 is a flow chart which illustrates a general tamperdetection operating scenario.

[0022]FIG. 2 is a flow chart which illustrates a tamper detectionoperating scenario according to the present invention.

[0023]FIG. 3 illustrates a noise-free, unmodified line drawingcorresponding to an image of an air field taken from space.

[0024]FIG. 4 illustrates a line drawing of a marked image resulting fromapplication of the present invention to the image illustrated in FIG. 3,i.e., a thumbnail of the original image is embedded in the originalimage.

[0025]FIG. 5a illustrates the marked image of FIG. 4 compressed withJPEG compression with a quality factor of 90.

[0026]FIG. 5b illustrates the difference between a thumbnail extractedfrom a decompression of the image of 5 a and the original thumbnail.

[0027]FIG. 5c illustrates malicious tampering of the image of FIG. 5a.

[0028]FIG. 5d illustrates results of semi-fragile tamper detectionaccording to the present invention applied to FIG. 5c.

[0029]FIG. 6 illustrates a histogram of the unenhanced difference imageillustrated in FIG. 5b, the compressed marked image.

[0030]FIG. 7 illustrates a histogram of the unenhanced difference imageillustrated in FIG. 5d, the modified marked image.

[0031]FIG. 8a illustrates the superimposition of noise on the image ofFIG. 5a.

[0032]FIG. 8b illustrates the difference image of FIG. 5a and FIG. 8a,i.e., the superimposed noise.

[0033]FIG. 8c illustrates the superimposition of noise on the modifiedmarked image of FIG. 5c.

[0034]FIG. 8d illustrates the result of applying semi-fragile tamperdetection according to the present invention to the image of FIG. 8c.

[0035]FIG. 9 is a block diagram which illustrates an embodiment of theimage tamper detection according to the present invention as anapparatus.

[0036]FIG. 10 is a block diagram which illustrates an embodiment ofthumbnail processing according to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0037] The following embodiments of the present invention are providedfor illustrative purposes only.

[0038] Referring now to the flow chart of FIG. 1, a general tamperdetection operating scenario is illustrated. In step 10, a sender uses atamper detection mechanism and a key to embed a tamper mark within animage. In step 11, the marked image is then compressed or exposed toadditive channel noise during transmission to a recipient, therebycausing distortion in the marked image during transmission. While enroute to a recipient, the image may also be intercepted by a maliciousparty who may employ image manipulation techniques to modify the imageinformation. This distorted, modified, and marked image is then passedalong to the recipient, who has no knowledge of the interception ormanipulation. Upon receipt of this image, in step 12, the recipientextracts the tamper detection mark and uses it in step 13 to determineif the image has been altered. If altered, a tamper alert is emitted instep 14 or the image is authenticated in step 15.

[0039] The semi-fragile detection system and method of the presentinvention is illustrated in the flow chart of FIG. 2 with the primaryfunctions of the system and method reflected in the following equations:

[0040] (1) In sub-process 21, the thumbnail of the entire original imageis computed. For demonstration purposes, consider an original N×N image,I, where the thumbnail is computed as:

T={t(j):j=0,1, . . . ,M}, M=(N/2^(n))².

[0041] This thumbnail contains the lowest order coefficients of ann-level wavelet decomposition, W, such that

T=W(I,n).

[0042] The user may select n to ensure that all important imagefeatures, those for which tamper detection is desired, are visiblewithin the thumbnail. A secondary consideration in the selection of thevariable n is the payload capacity of the original image.

[0043] (2) The marked image, G, is constructed by the data hidingencoder in step 22, that embeds T into the original image governed by akey k:

G=H(I,T,k)

[0044] The choice of H is made so that the size and dynamic range of theresultant marked image do not vary from the original image.

[0045] (3) The marked image may be exposed to various distortions,including image compression and noise from the transmission channel inprocess 23. Furthermore, a malicious attacker may manipulate the imagein step 24. The present claimed invention detects image manipulation astampering while neglecting effects of both modest image compression andchannel noise.

[0046] (4) The marked image received by the recipient is denoted as Ĝand is a version of G that may have been distorted and maliciouslymanipulated. From Ĝ, the embedded thumbnail of the original image, T, isextracted in step 25:

T=H ⁻¹(Ĝ,k).

[0047] (5) A new thumbnail is created from the received image in step26:

T=W(Ĝ,n).

[0048] (6) The two thumbnails are compared for similarity by taking theabsolute difference of corresponding inverse wavelet transforms in step27:

D=|W ⁻¹(T,n)−W ⁻¹(T,n)|.

[0049] (7) In the locations where D is large, the thumbnails are judgednot sufficiently similar, it can be assumed that tampering has occurred,and a tamper alert is emitted 28. Where D is small the thumbnails arejudged sufficiently similar, the effects are attributed to imagecompression and/or low-level channel noise and the image isauthenticated in step 29. If D is approximately equal to zero, thethumbnails are deemed sufficiently similar, the image is judged not tohave been altered in a significant manner, and the image isauthenticated in step 29.

[0050] In one exemplary embodiment of the present invention the datahiding technique used is Spread Spectrum Image Steganography (SSIS).However, any technique can be used that meets the data hidingrequirements of concealing a thumbnail in the original image so that itcannot be detected and neither the size nor the dynamic range of theoriginal image are changed. In this exemplary embodiment, thumbnailcomputation is accomplished using wavelet decomposition. However, anytechnique can be used that meets the thumbnail requirement of creating alow resolution counterpart of the entire image.

[0051]FIG. 3 illustrates an original image and FIG. 4 illustrates amarked image resulting from the semi-fragile system and method of thepresent invention applying a three-level wavelet decomposition to thisoriginal image to derive a thumbnail and then embedding this thumbnailin the original image using SSIS with a rate of 1/6 error correctingcode and an embedded signal power of 60. That is, the marked imageshould look very much like the original image with a mean-squared errorof 60 and a peak signal-to-noise ratio of 303.3 dB.

[0052] Referring now to FIG. 5a, the marked image illustrated has beencompressed with JPEG compression with a quality factor of 90 to resultin a compressed marked image of 2.89 bpp. To illustrate the semi-fragiletechnique of the present invention, the compressed, marked image of FIG.5a is decompressed, the thumbnail extracted and the thumbnail comparedto a newly computed thumbnail for the received image illustrated in FIG.5a. The difference image between the reconstructed and newly createdthumbnails is illustrated in FIG. 5b. From this difference image, thesubtle changes due to image compression can be seen, but grossabnormalities are not present.

[0053]FIG. 5c illustrates malicious tampering by manipulating thecompressed marked image of FIG. 5a by removing several planes on thetarmac and in the lower left image quadrant. FIG. 5c illustrates thereceived image. FIG. 5d illustrates the result of applying thesemi-fragile tamper detection technique of the present invention to themodified compressed marked image of FIG. 5c. The difference between thethumbnail extracted from the received image and the newly createdthumbnail for the received modified compressed image is illustrated inFIG. 5d. The significant differences between the two are evident in theclouds that occur in the locations of the removed planes. The effects ofcompression are still present in this difference but some are obscuredby the large difference caused by the manipulation.

[0054]FIG. 6 illustrates a histogram of the difference image used toproduce FIG. 5b, the compressed marked image. FIG. 7 illustrates ahistogram of the difference image used to produce FIG. 5d, the modifiedcompressed marked image. Large intensity values are present in themodified compressed marked image difference, e.g., 135, while thedifference values due to lossy compression are very small, e.g., 8. As aresult of this phenomenon, simple thresholding of the difference imageis used in one embodiment of the present invention to provide automatictamper detection.

[0055] Finally, as illustrated in FIG. 8a, adding channel noise with abit error rate equal to 10⁻⁴ to a marked image results in a slightlyaltered noisy image. The thumbnail for this marked image is extractedand compared to a newly constructed thumbnail for this image and thedifference image that results is shown in FIG. 8b. When the imageillustrated in FIG. 8a, a noisy marked image, is compressed andmaliciously manipulated, it becomes the compressed noisy modified markedimage illustrated in FIG. 8c. Applying semi-fragile tamper detectionaccording to the present invention yields the difference imageillustrated in FIG. 8d. In this difference image, noise is not readilyvisible while significant alterations are indicated by the clouds.

[0056] From the foregoing it will be obvious to one skilled in the artthat numerous modifications and variations can be made without departingfrom the spirit and scope of the novel aspects of the current invention.For example, FIG. 9 is a block diagram of a multiprocessor apparatuswith a shared memory that embodies the thumbnail computation process ofthe present invention. FIG. 10 shows another example, illustrating ablock diagram for an embodiment that employs individual units to performthumbnail processing. It is to be understood that no limitation withrespect to the specific embodiments illustrated is intended or should beinferred.

What is claimed is:
 1. A method for image tamper detection, comprisingthe steps of: (a) computing a thumbnail of an original image; (b)embedding the computed thumbnail in the original image to create amarked image; (c) transmitting the marked image to a recipient; (d)extracting the embedded thumbnail from the transmitted marked image; (e)computing a new thumbnail of the received image; (f) differencing theextracted thumbnail from the new thumbnail to create a difference imagethat represents the similarity of the two thumbnails; (g) alerting ifthe difference image shows the two thumbnails are not sufficientlysimilar; and (h) authenticating the image if the difference image showsthe two thumbnails are sufficiently similar.
 2. A method for imagetamper detection according to claim 1, wherein step (b) includesensuring that the embedding does not change the size of the originalimage.
 3. A method for image tamper detection according to claim 1,wherein step (b) includes ensuring that the embedding does not changethe dynamic range of the original image.
 4. A method for image tamperdetection according to claim 1, wherein step (b) further comprisesemploying a data hiding technique that is resistant to channel noise. 5.A method for image tamper detection according to claim 1, wherein step(b) further comprises employing a data hiding technique that isresistant to image compression.
 6. A method for image tamper detectionaccording to claim 5, wherein the image compression technique is JPEG.7. A method for image tamper detection according to claim 5, whereinstep (b) further comprises employing a data hiding technique that isresistant to channel noise.
 8. A method for image tamper detectionaccording to claim 1, wherein step (b) further comprises employing adata hiding technique based on Spread Spectrum Image Stenography.
 9. Amethod for image tamper detection according to claim 1, wherein saidthumbnail is defined as a low resolution version of an image.
 10. Amethod for image tamper detection according to claim 1, wherein steps(a) and (e) further comprise employing a wavelet decomposition.
 11. Amethod for image tamper detection according to claim 1, wherein step (g)further comprises threshholding the difference image to provideautomatic detection of tampering.
 12. An apparatus for image tamperdetection that creates and authenticates marked images based onthumbnail processing of that image, said apparatus comprising: (a) amemory having instructions stored therein, said instructions beingexecutable to perform a process of thumbnail computation; and (b) aprocessor comprising means for executing said instructions, saidinstruction comprising the operations of: (i) accepting an originalimage or a marked image; (ii) performing thumbnail processing of theaccepted image to: a. derive a thumbnail of the accepted image, b.create a marked image from the accepted original image and said derivedthumbnail, and c. authenticate an accepted marked image with saidderived thumbnail.
 13. The apparatus of claim 12, wherein said markedimage is created by embedding said derived thumbnail in the acceptedimage.
 14. The apparatus of claim 12, wherein said operation (ii)(3)includes extracting an embedded thumbnail from the accepted marked imageand comparing said derived thumbnail with said embedded thumbnail forsimilarity.
 15. A system for image tamper detection that creates andauthenticates marked images based on thumbnail processing of that image,said system comprising: (a) means for computing a thumbnail; (b) meansfor marking an original image with said thumbnail; and (c) means forauthenticating a marked image with said thumbnail.
 16. A medium thatstores instructions for image tamper detection that creates andauthenticates marked images based on thumbnail processing of that image,adapted to be executed by at least one processor to perform the stepsof: (a) accepting an original or marked image; (b) computing a derivedthumbnail from the accepted image; (c) for an accepted original image,embedding the derived thumbnail into the accepted image; (d) for anaccepted marked image, extracting an embedded thumbnail from the markedimage; and (e) for an accepted marked image, comparing said derivedthumbnail with said extracted thumbnail to determine similarity.